DO NOT MERGE: test graceful OOM shutdown via intentional memory leak by devin-ai-integration[bot] · Pull Request #969 · airbytehq/airbyte-python-cdk

devin-ai-integration · 2026-03-30T16:31:50Z

This PR targets the following PR:

feat(cdk): enable fail-fast shutdown on memory threshold with dual-condition check #962

Summary

Adds an intentional memory leak in AirbyteEntrypoint.read() to test the memory monitor's graceful shutdown behavior (introduced in #962). Each consumed record is duplicated 500× into a growing list to rapidly inflate Python process anonymous RSS, which should trigger the dual-condition fail-fast check.

This tests the positive case: the memory monitor should detect and raise AirbyteTracedException when both cgroup usage and process anonymous RSS exceed their respective thresholds.

DO NOT MERGE — test-only change to validate memory monitor behavior.

Companion PR: #970 (tests the negative case — cgroup pressure without process RSS growth)

Review & Testing Checklist for Human

Verify this PR is never merged to a release branch
Deploy a connector using this branch in a memory-bounded container and observe warning log at ~95% cgroup usage
Confirm graceful shutdown with AirbyteTracedException at ~98% cgroup + ~90% RssAnon
Verify queued state messages are flushed before the exception propagates

Notes

Pattern mirrors the approach from DO NOT MERGE: test graceful OOM shutdown #942 (previously validated)
This is one of two test branches off devin/1774478445-memory-failfast; the other (DO NOT MERGE: test dual-condition monitor with cache-only memory pressure #970) tests that non-process memory pressure does not trigger fail-fast

Link to Devin session: https://app.devin.ai/sessions/070ecb51ceee4f9189e1c09a83ba31cb

Co-Authored-By: patrick.nilan@airbyte.io <patrick.nilan@airbyte.io>

devin-ai-integration · 2026-03-30T16:31:52Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment and CI monitoring

github-actions · 2026-03-30T16:32:05Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1774888252-memory-failfast-feature-1#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1774888252-memory-failfast-feature-1

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/autofix - Fixes most formatting and linting issues
/poetry-lock - Updates poetry.lock file
/test - Runs connector tests with the updated CDK
/prerelease - Triggers a prerelease publish with default arguments
/poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
/poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

CDK API Reference

📝 Edit this welcome message.

Co-Authored-By: patrick.nilan@airbyte.io <patrick.nilan@airbyte.io>

github-actions · 2026-03-30T16:48:44Z

PyTest Results (Fast)

3 948 tests - 41 3 937 ✅ - 41 7m 28s ⏱️ -38s
1 suites ± 0 11 💤 ± 0
1 files ± 0 0 ❌ ± 0

Results for commit 7f906e9. ± Comparison against base commit 3c7c755.

♻️ This comment has been updated with latest results.

github-actions · 2026-03-30T16:52:22Z

PyTest Results (Full)

3 951 tests - 41 3 939 ✅ - 41 10m 50s ⏱️ -44s
1 suites ± 0 12 💤 ± 0
1 files ± 0 0 ❌ ± 0

Results for commit 7f906e9. ± Comparison against base commit 3c7c755.

♻️ This comment has been updated with latest results.

…-memory-failfast-feature-1

Patrick Nilan (pnilan) · 2026-03-31T21:52:06Z

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/23821153961

devin-ai-integration · 2026-03-31T22:01:10Z

DO NOT MERGE — test graceful OOM shutdown via intentional memory leak

This branch introduces an intentional memory leak (500x record duplication into a growing list) to test the CDK's dual-condition memory monitor fail-fast behavior. The corresponding source-faker PR (airbytehq/airbyte#75609) pins the CDK to the dev version from this branch.

Expected behavior: The memory monitor should detect cgroup usage ≥ 98% and anonymous memory (via cgroup v2 memory.stat anon field) ≥ 85% of current cgroup usage, then raise AirbyteTracedException for a graceful shutdown instead of an OOM kill.

CDK dev version: 7.13.0.post15.dev23821153961
Source-faker prerelease: publishing from PR #75609 (pinned to actor 14ea6dd8-83e4-44cc-91ea-c4e9dd8fa364 in @devin-ai-sandbox)

chore: initialize feature branch 1

d3056e3

Co-Authored-By: patrick.nilan@airbyte.io <patrick.nilan@airbyte.io>

devin-ai-integration bot mentioned this pull request Mar 30, 2026

DO NOT MERGE: test dual-condition monitor with cache-only memory pressure #970

Closed

5 tasks

DO NOT MERGE: add intentional memory leak to test graceful OOM shutdown

5aac720

Co-Authored-By: patrick.nilan@airbyte.io <patrick.nilan@airbyte.io>

devin-ai-integration bot changed the title ~~feat(cdk): memory fail-fast feature branch 1~~ DO NOT MERGE: test graceful OOM shutdown via intentional memory leak Mar 30, 2026

This comment was marked as outdated.

Sign in to view

octavia-bot bot temporarily deployed to PyPi March 30, 2026 17:07 Inactive

octavia-bot bot temporarily deployed to DockerHub March 30, 2026 17:07 Inactive

devin-ai-integration bot mentioned this pull request Mar 30, 2026

chore: DO NOT MERGE - source-faker with CDK dev image (memory leak test) airbytehq/airbyte#75609

Closed

3 tasks

Merge branch 'devin/1774478445-memory-failfast' into devin/1774888252…

7f906e9

…-memory-failfast-feature-1

octavia-bot bot temporarily deployed to DockerHub March 31, 2026 21:52 Inactive

octavia-bot bot temporarily deployed to PyPi March 31, 2026 21:52 Inactive

Patrick Nilan (pnilan) closed this Apr 2, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DO NOT MERGE: test graceful OOM shutdown via intentional memory leak#969

DO NOT MERGE: test graceful OOM shutdown via intentional memory leak#969
devin-ai-integration[bot] wants to merge 3 commits intodevin/1774478445-memory-failfastfrom
devin/1774888252-memory-failfast-feature-1

devin-ai-integration bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

devin-ai-integration bot commented Mar 30, 2026

Uh oh!

github-actions bot commented Mar 30, 2026

Testing This CDK Version

PR Slash Commands

Helpful Resources

Uh oh!

github-actions bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Patrick Nilan (pnilan) commented Mar 31, 2026 •

edited by github-actions bot

Loading

Uh oh!

devin-ai-integration bot commented Mar 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

devin-ai-integration bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Review & Testing Checklist for Human

Notes

Uh oh!

devin-ai-integration bot commented Mar 30, 2026

🤖 Devin AI Engineer

Uh oh!

github-actions bot commented Mar 30, 2026

👋 Greetings, Airbyte Team Member!

Testing This CDK Version

PR Slash Commands

Helpful Resources

Uh oh!

github-actions bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PyTest Results (Fast)

Uh oh!

github-actions bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PyTest Results (Full)

Uh oh!

This comment was marked as outdated.

Patrick Nilan (pnilan) commented Mar 31, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

devin-ai-integration bot commented Mar 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

devin-ai-integration bot commented Mar 30, 2026 •

edited

Loading

github-actions bot commented Mar 30, 2026 •

edited

Loading

github-actions bot commented Mar 30, 2026 •

edited

Loading

Patrick Nilan (pnilan) commented Mar 31, 2026 •

edited by github-actions bot

Loading